A Neural Reinforcement Learning Approach to Learn Local Dispatching Policies in Production Scheduling

Finding optimal solutions for job shop scheduling problems requires high computational effort , especially under consideration of uncertainty and frequent replanning. In contrast to computational solutions, domain experts are often able to derive good local dispatching heuristics by looking at typical problem instances. They can be efficiently applied by looking at few relevant features. However, these rules are usually not optimal, especially in complex decision situations. Here we describe an approach that tries to combine both worlds. A neural network based agent autonomously optimizes its local dispatching policy with respect to a global optimization goal, defined for the overall plant. On two benchmark scheduling problems, we show both learning and generalization abilities of the proposed approach. 1 Introduction Production scheduling is the allocation of limited resources to tasks over time, while one or more objectives have to be optimized. Many variants of the basic problem formulation exist , and most of them are NP-hard to solve [Pinedo, 1994], meaning that exact solution algorithms suffer from a non-polynomial increase of computation time. This constitutes a problem not only if the problem to solve surmounts a certain size, but also in moderately complex domains, where the occurrence of new or unexpected events-the arrival of new jobs or the breakdown of machines-makes frequent replanning necessary. Even more, technological changes like semiconductor fabrication or thin film production are posing additional challenges , since new problem structures-like conditional loops in the production process-occur, for which conventional optimization techniques may not be applicable. An alternative and far less time-consuming way is the application of simple heuristic dispatching rules that select the job to process next on an idle resource depending on the current situation. However, these dispatching rules only reflect heuristic knowledge and do not guarantee to lead to the optimal behaviour of the overall system. Even for experienced human experts it may become arbitrarily difficult to decide which dispatching rule to apply and how to time it in a certain scenario, since the effects on the dynamics of the overall system can hardly be predicted. 1.1 General Idea Here we propose an alternative way that allows to combine the desire for (nearly) optimal solutions with a time-efficient computation, provided by resource-coupled dispatching rules. The idea is to have learning agents, that are associated to each resource and determine the local dispatching policy. This policy is not fixed, but instead is autonomously learned by getting feedback …